feat(harness): introduce Multi‑Rollout parallel task execution for DeepAgent - #38
Conversation
…epAgent - Introduce MultiRolloutConfig, RolloutResult, selector strategies, and factory - Add MultiRolloutExecutor orchestrating clone → run → select → return - Hook DeepAgent.invoke() to delegate to MultiRolloutExecutor when enabled - Add workspace isolation via create_subagent() under sub_agents/ - Add strategy prefix injection for rollout diversity - Add 19 unit tests covering config, selectors, parallel execution, failure modes - Add full documentation (EN + CN) and navigation links
|
head_sha: 🤖 正在生成合并请求摘要，请稍候… |
|
head_sha: 🤖 AI 代码检视正在进行中，请稍候… |
|
head_sha: The pipeline(pipeline number:1977) is running. Please wait a moment... |
| results: list[Any] = [] | ||
| for i in range(n): | ||
| inp = self._copy_inputs(base_inputs) | ||
| strategy = variants[i % len(variants)] |
There was a problem hiding this comment.
head_sha: 355b68678b22d5eab4e0ab93959254a74cae3df2
🟡 Medium Priority
_build_attempt_inputs (executor.py:154) 执行 variants[i % len(variants)]。若用户将 strategy_variants 设为空列表 []，则 len(variants) 为 0，求模运算抛出 ZeroDivisionError。
此异常发生在 _build_attempt_inputs 中（在 _execute_parallel 的 try/except 保护范围之外），将导致整个 executor.invoke() 调用崩溃。虽然默认值提供 3 个策略，但 strategy_variants 是可配置字段，用户完全可能传入空列表。
建议：在 _build_attempt_inputs 开头增加空列表保护，或在 MultiRolloutConfig 的 __post_init__ 中校验 len(strategy_variants) > 0，在配置阶段尽早报错。
| if "query" in inputs: | ||
| inputs["query"] = query | ||
| elif "content" in inputs: | ||
| inputs["content"] = query |
There was a problem hiding this comment.
head_sha: 355b68678b22d5eab4e0ab93959254a74cae3df2
🔴 Critical
DeepAgent.invoke (deep_agent.py:2410) 调用 executor.invoke(invoke_inputs, session)，其中 invoke_inputs 是 InvokeInputs dataclass（@dataclass，不是 dict）。然而 MultiRolloutExecutor 的三个关键方法仅处理 dict：
_copy_inputs(executor.py:162-166)：isinstance(inputs, dict)对 InvokeInputs 为 False → 返回原对象（不复制），所有子智能体共享同一对象，存在并发竞态风险。_extract_query(executor.py:168-172)：对 InvokeInputs 走str(inputs)分支，产生类似"InvokeInputs(query='...', conversation_id=None, ...)"的无意义字符串。_set_query(executor.py:173-180)：对 InvokeInputs 不做任何事，策略前缀无法注入。- 最终
sub.invoke(InvokeInputs_object)中，子智能体的_normalize_inputs不处理InvokeInputs类型，直接抛出DEEPAGENT_INPUT_PARAM_ERROR。
后果：通过 DeepAgent.invoke 触发的 multi-rollout 路径完全不可用。唯一能工作的路径是直接使用 MultiRolloutExecutor 并传入 dict（如文档示例和单元测试所示），但这绕过了 DeepAgent 的正常入口。
| executor = MultiRolloutExecutor( | ||
| self, self._deep_config.multi_rollout | ||
| ) | ||
| return await executor.invoke(invoke_inputs, session) |
There was a problem hiding this comment.
head_sha: 355b68678b22d5eab4e0ab93959254a74cae3df2
🟠 High Priority
DeepAgent.invoke (deep_agent.py:2402-2405) 仅在 multi_rollout.enabled 为 True 时就创建 MultiRolloutExecutor，不检查 n_rollouts > 1。
而 MultiRolloutExecutor.is_enabled() (executor.py:57-58) 同时检查 enabled 和 n_rollouts > 1。当 enabled=True, n_rollouts=1 时，executor.is_enabled() 返回 False，走委托分支：
此时 inputs 是 InvokeInputs dataclass（由 DeepAgent.invoke 传入）。self._parent.invoke() 即 DeepAgent.invoke()，其中 _normalize_inputs(InvokeInputs) 因不处理该类型而抛出 DEEPAGENT_INPUT_PARAM_ERROR。
后果：enabled=True, n_rollouts=1 的配置本应等价于正常单路径执行，实际却抛出错误。用户若临时将 n_rollouts 调为 1 以禁用多路执行（如调试），会遇到非预期的异常。
| return await executor.invoke(invoke_inputs, session) | |
| if ( | |
| self._deep_config is not None | |
| and self._deep_config.multi_rollout is not None | |
| and self._deep_config.multi_rollout.enabled | |
| and self._deep_config.multi_rollout.n_rollouts > 1 | |
| ): |
| executor = MultiRolloutExecutor( | ||
| self, self._deep_config.multi_rollout | ||
| ) | ||
| return await executor.invoke(invoke_inputs, session) |
There was a problem hiding this comment.
head_sha: 355b68678b22d5eab4e0ab93959254a74cae3df2
🟡 Medium Priority
deep_agent.py 第 2400-2410 行的 multi-rollout 分支通过 return await executor.invoke(...) 提前返回，完全跳过了正常 invoke 路径中的以下关键逻辑（第 2412-2436 行）：
- AgentCallbackContext 生命周期：BEFORE_INVOKE / AFTER_INVOKE 回调钩子未触发。已注册的 AgentRail 回调（如安全护栏、审计日志等）将静默失效。
_invoke_active状态：标志位未被设为 True/False。is_invoke_active属性始终返回 False，_run_auto_invoke(第 2093 行) 的防重入检查可能被绕过。save_state(session)/clear_state(session)：父 agent 的会话状态不会被持久化或清理。invoke_inputs.result = result：返回结果未被写回 InvokeInputs 对象。
虽然文档声明"父工作空间不受影响"，但回调生命周期的静默绕过是一个破坏性行为——依赖回调的现有功能（如安全护栏、权限检查）在 multi-rollout 启用时会被跳过。
建议：在 multi-rollout 分支中至少触发 BEFORE_INVOKE / AFTER_INVOKE 回调（包裹 executor.invoke()），并正确管理 _invoke_active 和 session 状态。如果 multi-rollout 语义确实不需要完整生命周期，应在文档中明确说明哪些回调被跳过，并评估安全影响。
| self._deep_config is not None | ||
| and self._deep_config.multi_rollout is not None | ||
| and self._deep_config.multi_rollout.enabled | ||
| ): |
There was a problem hiding this comment.
head_sha: 355b68678b22d5eab4e0ab93959254a74cae3df2
🟠 High Priority
deep_agent.py 第 2402-2406 行的 multi-rollout 入口检查仅验证了 enabled，未检查 n_rollouts > 1。但 MultiRolloutExecutor.is_enabled() (executor.py 第 58 行) 同时要求 enabled 和 n_rollouts > 1。
触发路径：
- 用户设置
enabled=True, n_rollouts=1 DeepAgent.invoke()进入 multi-rollout 分支 (第 2402-2406 行)，创建 executor 并调用executor.invoke()executor.invoke()调用is_enabled()→ 返回False(因为n_rollouts=1)- executor 回退到
self._parent.invoke(inputs, session)→ 即同一个DeepAgent.invoke() - 回到步骤 2 → 无限递归，最终栈溢出
测试文件 test_enabled_requires_n_rollouts 仅验证了 executor.is_enabled() 返回 False，未覆盖从 DeepAgent 入口的完整路径。
建议：在 deep_agent.py 的 multi-rollout 入口检查中加入 n_rollouts > 1 条件，与 executor 的 is_enabled() 保持一致：and self._deep_config.multi_rollout.n_rollouts > 1
| ): | |
| if ( | |
| self._deep_config is not None | |
| and self._deep_config.multi_rollout is not None | |
| and self._deep_config.multi_rollout.enabled | |
| and self._deep_config.multi_rollout.n_rollouts > 1 | |
| ): |
|
head_sha: The pipeline(pipeline number:1977) is running. Please wait a moment... |
2 participants
Paired: GitHub #38 ↔ GitCode !1977
What type of PR is this?
/kind feature
What does this PR do / why do we need it
This PR introduces Task‑Layer Multi‑Rollout, a new parallel‑execution mechanism for DeepAgent that allows multiple independent strategies to be explored simultaneously for a single task.
The problem
A single agent execution trajectory can get stuck in a local optimum. For complex coding tasks (e.g., hard bug fixes), the agent’s first strategy is often not the best one. Restarting the entire task manually is slow and wastes the context already built up.
Auto‑Harness Best‑of‑N solves this for CI repair, but there was no mechanism for task‑level strategy exploration during normal DeepAgent.invoke().
The solution: Multi‑Rollout
When enabled, DeepAgent.invoke() transparently switches to a multi‑attempt pipeline:
Spawn N subagents with isolated workspaces
Inject different strategy prompts into each attempt
Run all attempts in parallel
Collect RolloutResult(success, exception, output_text)
Select the best result via a pluggable selector
Return the winning output to the caller
This gives the agent multiple “shots” at the same task without losing context or requiring manual restarts.
How it works
Invoke path
Workspace isolation
Each subagent is created via:
Each attempt receives its own workspace under
sub_agents/.Strategy diversity
Each attempt receives the same task, prefixed with a different strategy instruction:
correctness‑focused
minimal‑diff
edge‑case‑focused
Real divergence also depends on LLM temperature > 0.
Selectors
Three built‑in selection strategies:
first_successful — fastest, safest default
longest_output — prefers completeness
shortest_output — prefers minimal diffs
Files changed
agent-core
Caveats
Streaming: Multi‑rollout works only with
invoke(), notstream().Cost: n_rollouts = 3 → ~3× LLM cost.
Workspace state: Parent workspace is untouched; caller must copy files if needed.
How to enable
Via DeepAgentConfig
Standalone executor
Tests
19 unit tests covering:
Config defaults and validation
All selector strategies
Factory error handling
Disabled path (delegates to parent)
Parallel spawn + selection
Partial failure recovery
Complete failure propagation
Strategy prefix injection
Self-checklist
[x] Design: Reviewed with maintainers
[x] Test: 19 unit tests added
[x] Verification: Parallel attempts validated across multiple task types
[ ] Interface: No external API changes
[x] Document: Full docs added in EN + CN